Measuring differential item and test functioning across academic disciplines

نویسنده

  • Judith Runnels
چکیده

Differential item functioning (DIF) is when a test item favors or hinders a characteristic exhibited by group members of a test-taking population. DIF analyses are statistical procedures used to determine to what extent the content of an item affects the item endorsement of sub-groups of test-takers. If DIF is found for many items on the test, the final test scores do not represent the same measurement across groups in the population of test-takers. This is known as differential test functioning (DTF). DTF is of particular concern in tertiary level language tests, where test-takers often differ in academic discipline. This study examined the DIF and DTF of an in-house developed assessment designed to measure how well first year students of five academic disciplines achieved material over the course of a year of English language study. The DIF and DTF tests were performed using Rasch analysis, which controls for ability across groups, ensuring that items are only flagged if groups of test-takers of the same ability levels exhibit a significantly different probability of endorsing the item. The current analysis outlines the process for checking for DIF and DTF and finds that even though DTF is unlikely, there were several items that favored and hindered some majors. Recommendations for modification of items are made and the importance of establishing a process to check for DTF and DIF, especially when the test-takers are from different disciplines of study, is discussed.

برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

ثبت نام

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

منابع مشابه

Gender-based DIF across the Subject Area: A Study of the Iranian National University Entrance Exam

This study aimed at investigating differential item functioning (DIF) on the Special English Test of the Iranian National University Entrance Exam (INUEE). The effect of gender and subject area was taken into account. The study utilized one-parameter IRT model with a sample of 36000 students who sat for the INUEE Special English Test in 2004 and/or 2005. The findings confirmed the presence of D...

متن کامل

Selecting the Best Fit Model in Cognitive Diagnostic Assessment: Differential Item Functioning Detection in the Reading Comprehension of the PhD Nationwide Admission Test

This study was an attemptto provide detailed information of the strengths and weaknesses of test takers‟ real ability through cognitive diagnostic assessment, and to detect differential item functioning in each test item. The rationale for using CDA was that it estimates an item‟s discrimination power, whereas clas- sical test theory or item response theory depicts between rather within item mu...

متن کامل

Differential Item Functioning (DIF) in Terms of Gender in the Reading Comprehension Subtest of a High-Stakes Test

Validation is an important enterprise especially when a test is a high stakes one. Demographic variables like gender and field of study can affect test results and interpretations. Differential Item Functioning (DIF) is a way to make sure that a test does not favor one group of test takers over the others. This study investigated DIF in terms of gender in the reading comprehension subtest (35 i...

متن کامل

Differential Item Functioning and Unidimensionality in the Pearson Test of English Academic

Since the Pearson Test of English Academic (PTE Academic) was designed to assess skill differences among test-takers at all points along the ability continuum, rather than to determine cutoff scores, it is important to examine the extent to which the instrument assesses what it is intended to measure (validity) as well as the extent to which the test is consistent (reliability) in measuring ELL...

متن کامل

A confirmatory study of Differential Item Functioning on EFL reading comprehension

The  present  study  aimed  at  investigating  DIF  sources  on  an  EFL  reading  comprehension test.  Accordingly,  2  DIF  detection  methods,  logistic  regression  (LR)  and  item  response theory  (IRT),  were  used  to  flag  emergent  DIF  of  203  (110  females  &  93  males)  Iranian EFL examinees’ performance on a reading comprehension test. Seven hypothetical DIF sources were examin...

متن کامل

ذخیره در منابع من


  با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید

برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

ثبت نام

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

عنوان ژورنال:

دوره   شماره 

صفحات  -

تاریخ انتشار 2013